Add MCP server endpoint for AI agent doc access - #2727

- Add error boundary in /mcp endpoint to catch JSON parse errors (-32700)
- Return JSON-RPC error instead of HTML 500 for parse failures
- Handle unknown tool names with -32602 error instead of returning nil
- Add generic error handler in Mcp::Server.handle for unexpected exceptions
- Fixes issues where malformed requests would return HTML error pages

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Validate all arguments against tool inputSchema before dispatch
- Check required fields are present
- Validate field types (string, integer, number)
- Enforce min/max constraints on numeric values
- Return -32602 (invalid request) for validation failures
- Add comprehensive tests for missing/invalid arguments and type mismatches

Fixes comment about params being dereferenced without validation.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Replace simple Node#text concatenation with proper block element handling
- Add line breaks before/after block elements (p, div, h1-h6, ul, ol, li, blockquote, pre, br)
- Preserve meaningful whitespace while removing excess blank lines
- Prevents adjacent block elements from concatenating without separation

Fixes issue where <h1>Title</h1><p>Body</p> would become TitleBody.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Add offset and limit parameters to search (same as list_docsets)
- Validate that query is non-empty to prevent matching everything
- Return paginated results with metadata (offset, limit, total, returned)
- Prevents large result sets from exceeding agent context limits
- Add tests for pagination and empty query validation

Fixes issue where large docsets could generate oversized responses.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Add DB_CACHE to cache parsed db.json files in memory
- Load database once per docset and reuse across requests
- Prevents reparsing multi-megabyte files on every page lookup
- Reduces memory allocations and CPU overhead on concurrent calls
- Cache key includes docs_path to handle test/production separation

Fixes performance issue with repeated full-database parsing.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Verify malformed JSON returns -32700 parse error
- Ensure /mcp endpoint returns JSON-RPC error instead of HTML 500

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Entry paths carry a #fragment for entries that share a page (377 of 508
bash entries), but db.json is keyed by the page path alone, so the
search -> get_page workflow failed with "Page not found".

Their tags were missing from block_element?, so tables and definition
lists - most of the CSS and HTML reference content - collapsed into runs
like "NameTypefooString".

Nokogiri's #traverse is post-order, so a block element's newline was
emitted after its text and text preceding the block ran into it
("Options are:one\ntwo").

Whitespace was collapsed over the whole document, flattening the
indentation of <pre> blocks and corrupting code samples. Text is now
collected in runs of equal preformattedness and only the ones outside
<pre> are collapsed.

Every db.json value is also on disk as the .html file the app serves, so
get_page reads that (~0.01ms) rather than parsing up to 100MB of JSON.
This drops DB_CACHE, which pinned every db.json touched in memory
(571MB across the docsets here) and served stale text after a re-scrape.
The path comes from the caller, so it is now kept inside the docset.

index.json was re-read and re-parsed on every search (37ms for the
largest one here). It is now cached per docset, keyed on mtime and size
so a re-scrape is picked up. The indexes are small - 16.5MB for all 48
docsets here - unlike the db.json cache this replaces.

The loop above it already rejects every field that the schema does not
declare, whatever additionalProperties says.

An array or scalar payload made the rescue handler itself raise a
TypeError on request['id'], which escaped handle and surfaced as -32603
from the route. Batches and scalars now get -32600.

A request without an id is a notification and must not be answered, but
notifications/initialized - which every client sends right after the
handshake - was answered with -32601 and a null id. Notifications now
get 202 with an empty body.

A tool that cannot answer - an unknown slug, a missing page, an empty
query - now returns its message as an isError result, which is what the
MCP spec asks for: a protocol error is handled by the client and never
reaches the model.

The assertions sat behind "if response.key?('error')", so the test
passed without checking anything if the error stopped being returned.